(1) Field of the Invention
The present invention relates to a motion-compensated interframe coding system for coding a television picture signal under the compensation of the coded signal in response to the variation of the television picture, which is caused by the motion thereof, on account of the motion vector representing the shift of the picture between adjacent frames thereof, particularly, to the improvement of the above system.
(2) Description of the Prior Art
The recently accustomed digitization of the television picture signal has been adopted in various kinds of equipments used for the television broadcast, for instance, a video tape recorder because of the easiness of the time base correction and the freedom from injurious noise which can be attained by the digitization thereof. However, the digitization of the television picture signal requires the extremely broadened frequency band. That is, the bandwidth required for the analog colour television signal of the NTSC system is 4.5 MHz, whilst the bandwidth required for the ordinary digitization thereof is 90 MHz. So that, the frequency band compression of maximally high degree is required for the digilized colour television signal.
On the other hand, the considerably high redundancy can be found in the television picture signal. Accordingly, the considerably large amount of the information to be transmitted can be reduced for the digitized television picture signal by utilizing the above redundancy skillfully.
The television picture signal coding systems having the high efficiency as mentioned above can be classified into two groups, that is, such a group of coding systems for treating the picture signal of each frame individually as the DPCM system or the Hadamard conversion system and such another group of coding systems for treating picture signals of plural successive frames as a whole as the so-called interframe coding system.
In the interframe coding system for the television picture signal, the above redundancy is utilized in the direction of time axis thereof. The television picture signal has the considerable similarity between adjacent frames. Particularly, the still picture signal has the sameness between adjacent frames. Accordingly, after the completely coded picture signal of the first frame has been transmitted, the remaining picture signals of the succeeding frames can be transmitted by coding only the differences between adjacent frames with the high efficiency and the sufficient quality in the interframe coding system.
However, the moving picture signal has a considerable amount of differences between adjacent frames, so that the increase of the informations to be coded which is caused by the motion of the picture is the sore spot of the interframe coding system.
Nevertheless, a fair amount of correlation between adjacent frames can be found even in the moving picture signal. Particularly, it can be found frequently in the moving picture that a small block thereof is merely shifted between adjacent frames. Accordingly, in case the simple correlation between adjacent frames is obtained after the above shift is compensated in response to the motion of picture detected between adjacent frames, the amount of significant differences between adjacent frames can be reduced remarkably even in the moving picture signal, so that the interframe coding can be effected with an extremely high efficiency for the moving picture signal.
An ordinary circuit configuration of the motion compensated interframe coding system based on the detection of the motion vector representing the motion of picture is shown in FIG. 1. In the sending end of the system shown in FIG. 1, an incoming picture signal is applied to a quantizer 2 through a subtracter 1. The quantized picture signal is applied through an adder 3 to a predictor 4 which comprises a frame memory 5, a motion detector 6 and a motion compensator 7 for detecting the motion vector between the present frame and the immediately preceding frame. The motion vector derived from the motion detector 6 is multiplexed to the quantized picture signal by a multiplexer 8, so as to be transmitted to the receiving end, as well as the quantized picture signal derived from the frame memory 5 is compensated by the above motion vector in the motion compensator 7, so as to form an interframe predicted signal which should be subtracted from the incoming picture signal in the subtractor 1.
In the receiving end, the preceding frame picture signal derived from a frame memory 11 is compensated in a motion compensator 12 by the motion vector separated from the received picture signal by a demultiplexer 9, so as to reproduce the same interframe predicted signal as in the sending end. The reproduced interframe predicted signal is added in an adder 10 to the interframe differential signal derived from the demultiplaxer 9, so as to restore the original picture signal.
The motion compensation based on the detected motion vector as mentioned above can be utilized for the following various applications:
(i) Application to the industrial motion measurement for detecting an intricate motion of a moving body. PA1 (ii) Application to the correction of the vibration of picture which is caused equally in the whole area by the vibration of a camera and accordingly can be detected easily by detecting the partial motion. PA1 (iii) Application to the noise reduction of the picture signal owing to the contrast between the rich correlation of the picture signal and the poor correlation of the noise.
The aforesaid motion compensation based on the detected motion should be applied effectively in particular for the noise reduction of the moving picture signal because of the comparatively poor correlation thereof. However, the direction and the velocity of partial motion of the moving picture can be detected hardly by the conventional method for detecting the motion vector.
The following method is well known for detecting the partial motion of the television picture or the televised cinema film and will be explained by referring to two adjacent frames as shown in FIG. 2.
By the above well known method, a block having an appropriate size is selected in the present frame of the above two adjacent frames. The above selected block is assumed to contain n picture elements having numbers 1, . . . , n and signal levels B.sub.1, . . . , Bn respectively, where the symbol Bi denotes the signal level of the picture element i. Similarly, another block having the same size is selected in the immediately preceding frame, and is assumed to contain n picture elements having respectively voltage levels Bi similarly as mentioned above. Under these assumptions, the correlation C between those two blocks can be obtained by the calculation according to the following equation. ##EQU1##
That is, the above calculation is carried out repeatedly with regard to the variously shifted positions of the block selected in the preceding frame respectively, so as to obtain the position thereof which presents the strongest correlation to the block selected in the present frame. As the result, the motion vector of the picture in the present frame can be obtained as the difference between the position vector in the present frame and that in the preceding frame.
The mentioned above will be explained more concretely by referring to the following example, that is, the motion of picture which is represented by three circles (n-1), n and (n+1), the positions of which are shifted successively between three adjacent frames n-1, n and n+1, as shown in FIG. 3.
Nextly, FIG. 4 shows the case that the motion is detected between the frames n-1 and n. In FIG. 4, the solid rectangle represents the size of the block selected for detecting the motion of picture, the solid circle represents a picture of a body shown in the preceding frame, the chain-lined circle represents a picture of another body, and, the above mentioned solid rectangle represents the block attended in the present frame. Regarding a position of the last rectangle, nine blocks, which are shifted in eight directions, namely, upward, downward, left, right and four oblique directions and positioned at the center (not shifted), are settled in the preceding frame, so as to obtain the motion vector based on the position of the settled block having the highest conrrelation to the original solid rectangle. In FIG. 4, a unit of shift is represented by a mark of .vertline.--.vertline. similarly as hereinafter, and a dotted big line shows the above block having the strongest correlation, and further arrow marks show that plural kinds of motion vectors are obtained as an inaccurate result of the above detection of motion.
Similarly as mentioned above, FIG. 5 shows the case that the motion is detected between the frames n and n+1, in which case the same as above is repeated with a similarly inaccurate result of the detection of motion. In FIG. 5, a dotted thin line surrounding the blocks indicates a range in which those blocks can be positioned.
In the above examples of the conventional method for detecting the motion, the correct motion vector is that shown by the big arrow mark only, and the correlation should be examined regarding nine blocks, so that the obtained result of the detection of motion is insufficient. Moreover, the simply increased number of blocks to be examined regarding the correlation cannot assure the possibility of the detection of correct motion. That is, in order to detect the sufficiently correct motion, it is required to settle a large number of blocks in the whole area of the frame to be examined. Even if a practically possible number of blocks is restricted, it is impossible in practice to examine the correlations regarding all of those blocks in a certain desired time duration.
According to the above mentioned method for detecting the motion vector, the pattern recognition regarding a body in the picture is not required at all, so that such an advantage as the motion vector can be detected mechanically. Contrarily, according to the above method, as mentioned above, an extremely broad area and an extremely large number of blocks to be examined for detecting the motion vector, so that it is not realizable to detect the motion vector at the real time because of the time duration required for the calculation of the above method. Accordingly, it is an inevitable subject for detecting the motion vector how to detect the more correct motion vector based on the less number of times of calculations for examining the correlations within a desired time duration. Therefore, various methods for accomplishing the above subject have been investigated, so as to improve the above conventional method for realizing the possibility of the real time detection of motion.
One of those improved method for detecting the motion vector will be explained hereinafter.
The above improved method for detecting the motion vector is based on the fact that, in the case that, after the motion between the frames n-1 and n has been detected, the motion between the frames n and n+1 will be detected, if the positions of the blocks settled in the frame n are shifted previously by a distance corresponding to the already detected motion in the opposite direction from the reference position thereof, the correlation can be examined between the blocks settled in an area having the highest probability and the original block the motion vector relating to which should be detected, so that, regardless of the limitation of the number of times of the calculation for examining the correlation, the more correct motion vector can be obtained.
Nextly, in the case the above improved method is applied to the detection of motion as shown in FIG. 3, the detection of motion in the first step, that is, the detection of motion between the frames n-1 and n is performed similarly as mentioned above. However, in the second step in which the detection of motion between the frames n and n+1 is performed by referring to the calculated result for the above detection of motion in the first step, the position of the block in the frame n, which block is compared with the attended block (the solid rectangle) in the frame n+1 for calculating the interframe correlation, is shifted previously by a distance corresponding to the motion detected between the frames n-1 and n in the opposite direction from the reference position as shown in FIG. 6. In FIG. 6, the reference position is shown by a double dotted chain line, and the shift vector is shown by a dotted double arrow mark. According to the mentioned above, regardless of the same number of blocks to be examined regarding the correlation as according to the conventional method shown in FIG. 4, that is, nine blocks, it can be recognized that the correct motion between the frames n and n+1 can be detected. The motion between the frames n+1 and n+2 and so on can be detected correctly by repeating the similar calculation as mentioned above.
According to the above improved method, the detection of motion in the first step is not sufficiently correct, whilst that in the second step and so on is correct. On the other hand, if the detection of motion in the second step is attended, it is apt to be considered that it is possible according to the aforesaid conventional method to examine the twice number of blocks in the twice longer time duration and, as a result, to obtain the same effect as according to the improved method. However, the above consideration is mistaken as the following example shows.
FIG. 7 shows the case that eighteen blocks are examined between those two frames, and the area surrounded by a dotted big line shows the range in which the correlation is examined. It is certified in this case that the motion can be detected correctly. However, if the motion is performed as shown in FIG. 8, such a case can take place that no more correct motion is detected as shown in FIG. 9. In contrast therewith, it is possible according to the above improved method to detect the correct motion in a half time duration of that according to the conventional method, as shown in FIG. 10.
Apparently from the mentioned above, according to the improved method, the number of correlations to be calculated at every frame is not so many, so that it is possible to detect the more correct motion vector at real time. However, the above improved method has such a further defect as follows:
Generally speaking, in order to detect the motion vector, at first, the correlation of the pictures between two adjacent frames is detected according to any one of the following three methods, and then the motion vector of the picture is detected by referring to the strength of the detected correlation. According to the first method, the weaker the square correlation of signal levels of respectively corresponding picture elements between two adjacent frames is, that is, the smaller the square of the signal level differences between those picture elements is, the stronger the correlation of the picture signals between those two frames is. According to the second method, the smaller the absolute value of the difference of signal level between those picture elements is, the stronger the correlation of the picture signals between those two frames is. Furthermore, according to the third method, the lesser the number of picture elements the differences of signal levels of which exceed the appropriately settled threshold level is, the stronger the correlation of the signal levels between those two frames is.
However, according to those method for obtaining the square correlation or the absolute value correlation, in the case that, even if a specified target body having a signal level which is extremely different from those of surrounding portions moves in a picture frame, the specified target body is small, it is feared that the motion thereof is taken hold of as a whole motion of blocks provided in the picture frame for examining the above correlation of picture elements between those blocks. The reason thereof is that the correlation regarding the whole blocks is decided by the difference of picture signal levels in the portions occupying respectively large areas of those blocks. On the other hand, according to the above method employing the threshold level, such an advantage can be obtained that the circuit configuration required for detecting the motion vector is simplified, as well as it is impossible to detect the motion vector regarding a picture having no signal levels which exceed the threshold level. Consequently, all of those three conventional methods for detecting the motion vector have such defects as the motion of contents of the picture frame cannot be detected sufficiently.